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ABSTRACT 



The possibility of creating an appropriate online learning 
environment for distance adult students was examined in a study of 22 
Pennsylvania State University (Penn State) students ' attitudes regarding the 
online version of a course offered as part of Penn State’s masters of 
education program. The students completed surveys before, during, and after 
the course. The survey questions focused on technology use, the learning 
process, the course's structure and content, and students’ opinions regarding 
how well they had met the course objectives. In addition, 16 paired responses 
from the precourse and postcourse surveys were analyzed using the Wilcoxon 
matched-pairs signed ranks test. Most students rated their mastery of course 
objectives very highly. Eighty percent of the students interacted with other 
students between four and five times each week. Sixty percent stated that 
they interacted with the instructor "very often or always." As the course 
progressed, students reported significantly more expertise in computer use 
and less apprehension about using the Internet for the class. Students stated 
that the course's greatest strengths were the course instructors and the 
guidance and support provided. The surveys also established that the learning 
environment created made good matches to the learning principles that had 
guided the delivery design. (Contains 25 references and 5 figures.) (MN) 
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A dominant response by higher education institutions to their changing 
environment is the development of online distance education programs. 
This paper reports on a program-based initiative that introduced a group 
of adult students to masters-level study in a computer-mediated online 
environment. Their experience, monitored by means of surveys, 
evaluative comments and time tracking charts, is reported on. The results 
indicate that a collaborative learning environment was created. Small 
group and paired work was important in this development. In addition, 
students perceived an increase in their ability to use technology. Results 
also indicate students moved towards independence and interdependence 
within the course environment. 
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Many universities are wishing to respond to the changing characteristics of their 
students and the increasing access to technology for both faculty and students. 
University administrators and planners often see new technologies as opening up 
new markets of students who traditionally might not have had access to 
undergraduate and graduate education. Between 1995 and 1998 the number of 
courses offered at a distance doubled and the percentage of higher education 
institutions offering distance education courses in the USA increased by one third 
(National Center for Educational Statistics 1999). 

Distance education 'is planned learning that normally occurs in a different place 
from teaching' (Moore and Kearsley 1996, p 2). The National Center for Educational 
Statistics (NCES), through its PEQIS (Post Secondary Quick Information System) 
surveys, provides a picture of trends in distance education in the USA, using a more 
constrained definition - one that focuses on electronic media. NCES defines distance 
education as education and training courses delivered to off-campus students via 
audio, video or computer technologies. The most recent report notes that several 
forces are at work - convergence of communication and computing technologies, 
changing demographics of students, and the need to reduce costs. Institutions, it is 
suggested, see distance education as attractive as they seek increased enrolments and 
decreased costs. Students, the survey says, benefit from increased access for those 
constrained by geography, time, job and family responsibilities and finances. 

Distance education offerings are increasing. More courses are being offered, 
particularly by institutions who already have a distance education tradition. 

Several writers and researchers also suggest that computer-mediated communication 
(CMC) will enable teachers and institutions to extend educational and 
communication opportunities (eg Dede 1996; Spencer 1998). Hiltz (1994) and Kaye 
(1989) claim that CMC is able to support teaching and learning for distance and 
dispersed learners and build the type of community that strongly supports learning. 
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A new educational paradigm that is time independent, place independent and 
supports many-to-many interactions is heralded by some writers (Beller 1998; 
Harasim 1989; Kaye 1989; Turoff 1997). For faculties using these technologies for the 
first time, there are new issues related to teaching, learning, and learner 
characteristics to be considered as they come to meet the personal challenges of 
teaching in a new way. Windschitl (1998) has urged university researchers to become 
involved in researching instruction using the internet to explore emerging 
possibilities for improved instruction at all levels. 

Research background 

Two broad questions formed the foundation for a review of theory and research: 
What are the characteristics of adult learners that should be developed, encouraged, 
and taken advantage of in instruction offered through distance education? What are 
the elements of an effective online learning environment? These two questions were 
set within the context of a course on research methods that focused on developing an 
online learning community for graduate students enrolled in a web-based Masters 
degree program in Adult Education. 



Graduate students as adult learners 

In writing about part-time graduate school study. Pitman (1997, p 72) comments 
about adult students: 

Adult students generally want to be treated as adults, a preference that 
grows stronger over time. Adults tend to be more goal oriented than 
younger students, and they look for immediate opportunities to apply 
course content. Also, they bring valuable experience to the classroom; 
professors and other students learn from them. All of this is grossly 
oversimplified, of course, but it is generally helpful in designing 
educational programs for adults. 

Other broadly useful design parameters as characteristics of adult learners are the 
notions of self-direction and a wish to be active participants in their learning. 
Brookfield (1986) and Knowles (1980) both support the ideas that adults wish to 
participate in their own learning and assume some responsibility for it. In a 
graduate-level course there can be some tension between the formal framework of a 
course and these characteristics. In addition, the course instructor has to recognise 
the points at which support or guidance is needed and provide it in a manner that 
does not undermine the learner. 

Group work and collaboration have been widely used as a means of sharing 
experience, exploring ideas and building new learning (Tennant 1991). Activities in 
small groups and with learning partners can develop a sense of community that 
supports learning (Cook 1995). The development of self-direction and collaboration 
in distance education needs support and guidance, and providing this becomes an 
important role for the instructor (Eastmond 1997). 

Distance study at a graduate level has strong links to adult learning. It is increasingly 
through distance education that an adult returns to school. The need for an advanced 
degree to further work aspirations or a return to study as a preparation for a re- 
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focused stage of life often lead adult students to graduate school (Pitman 1997). In a 
survey of graduate students, Hammon and Albiston (1998) found study is usually 
undertaken on a part-time basis. Family and work constraints were factors 
considered in the decision to return to study. The challenge is for higher education to 
adapt objectives, content and presentation to support learning for this group. 
Graduate-level study calls for attributes of independence and focused effort. 
Dialogue and debate are expected (Swenson 1995). 

Learning environments 

Traditionally most learners have assembled together in classrooms of some type. In 
these settings, professors aim to guide the learning activities that help students 
construct knowledge. Typically this guiding process involves three key steps: 

• Understanding and knowing the prior knowledge of the students so links can 
be made between old learning and new; 

• Responding to the students to promote feedback and gathering the responses 
to use in the construction of new or refined knowledge; 

• Helping the students see how activities and ideas presented build or combine 
to increase understanding and knowledge. (Slavin 1997) 

Such a constructivist approach emphasises the active role learners take in knowledge 
construction. Learning thus defined has strong social aspects, and interactions 
between learners and instructor, and learners and learners, are important. This type 
of learning environment has been valued in the adult education program that is the 
context for the research reported here. Maintaining such an approach in a distance 
education environment was considered important. 

Online groups/learning communities 

The social dimension of learning and social interactions can have a significant impact 
on learning outcomes (Burge 1994; Jonasson et al 1995). Learners need the chance to 
interact, discuss, refine and reflect. Collaboration allows them to do this. Computer- 
based collaboration adds flexibility with its any time, any place capability (Harasim 
et al 1995). It also makes collaborative discussions, peer activities and small group 
work possible. These activities encourage independence and self-direction and 
demonstrate the interactivity that is the major advantage of computer conferencing 
(Eastmond 1998; Mason 1994; Schrum 1998). A key role for the instructor is to act as a 
facilitator and to carefully monitor and support the interactions (Burge 1994; Wegerif 
1998). A skilled instructor (or facilitator) seems to be required to ensure that groups 
work well together (Gunawardena and Zittle 1997; Wegerif 1998). 

Study context 

This study is based on the results of four surveys, time tracking charts and 
evaluations used to gain students' responses to the online teaching and learning 
environment that was developed for them. The theoretical perspective reflects the 
importance of developing an online learning community for graduate students, 
emphasising collaboration and self-direction in learning. 




The graduate students in this study were a 'typical 7 adult group. Many were 
returning to study, most were women, and all had other responsibilities. Twenty-six 
students logged on at the beginning of the course. After the first week, when two 
students withdrew, the group settled to 24 students. Later in the course, two students 
decided to defer completion of the course due to family responsibilities. The students 
were advised in advance that they should allow 12 hours per week for their 
coursework and have access to the computer equipment required for the distance 
education course that had no face-to-face component. 

The online Masters degree program is offered from a land grant university in the 
USA (Penn State) that has a history of outreach and distance education. Recently, in 
response to the changes in student demographics and needs, the university has 
developed a virtual campus (known as the World Campus) that is designed to 
continue the outreach mission and extend that commitment to distance students. 
Central to the virtual campus development has been a focus on using new 
technologies to provide an interactive environment for students. 

Although the course was an existing one that had been previously taught by the 
researcher, careful planning and preparation occurred before it was placed online. A 
team approach was employed as for all courses on the World Campus, involving an 
instructional designer, programmers, program manager, marketing and research 
specialists. (Faculty members are released from one course for the two semesters 
prior to instruction for course development. Whenever they teach for the World 
Campus, they are also released from teaching a course on campus. Thus, all 
instruction through the World Campus is part of a faculty member's load.) An 
additional instructor was added to assist with grading some assignments as a way to 
prepare him for teaching upcoming courses through the World Campus. 

The online version of the course was developed using WebCT to create a password- 
protected web site whose major features were multiple bulletin boards, an email 
system, automatic record-keeping available to faculty and individual students at any 
time, online text and images, and links to other web sites. Students were provided 
with a weekly schedule for course task completion. Assignments included an 
approximately even balance of those to be completed individually (and submitted to 
the instructor by email or done as a self-check) and those that required small group 
work that was posted to the appropriate bulletin board. 

Methods 

Data were gathered before, during and at the end of the course. There were four 
major survey points. A pre-course survey was administered online at the beginning 
of the course. The questions focused on technology use, the learning process, the 
course structure and content. The post-course survey repeated the same questions 
and also included two questions related to the time tracking exercise. A mid-course 
survey (referred to as 'mid- survey 7 ) focused on the course objectives, feedback, 
assignments, support material and faculty/ student interactions. Students were: 
given the opportunity to add further comments; asked to indicate their willingness to 
take another course designed in a similar way; and asked whether they would 
recommend the course to others. The reflection survey (referred to as 'reflect survey 7 ) 
asked students how well they felt they had met course objectives. It also asked for 
comments about teamwork and the development of an online community. In 
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addition to the surveys, the students were asked to keep a weekly Time Tracking 
Chart and to post evaluative comments on group work to a discussion forum. A 
Suggestion Box was available throughout the course for student input and feedback. 

The surveys used a Semantic Differential approach that asked for students to react 
on a seven-point scale to polar opposite adjectives about specific content (Osgood et 
al 1967). The pre and post surveys were used to determine changes in student 
response over the duration of the course. The two remaining surveys and the Time 
Tracking Chart provided additional data relevant to the pre-post survey themes. The 
evaluative comments were used to gain an overall understanding of the students' 
experiences. The qualitative data were then analysed to identify themes in the 
comments that develop a richer picture of the student experiences. According to 
Windschitl (1998), using mixed methods, as in this study, is superior in identifying 
and describing the impact of the internet on instruction. 

From the pre and post survey, sixteen paired responses were analysed using the 
Wilcoxon matched-pairs signed ranks test to determine if there was a statistically 
significant change in responses to the pre and post items. For all items in the 
technology use theme, a directional change was hypothesised. It was considered that 
students would report: moving toward expertise in computer use (Q2); less 
apprehension about internet use for the course (Q6); and to increasingly like using 
the computer (Q14). One tail tests were used for these items, and also for two items 
in the course and content theme. It was hypothesised that students would report 
increasing expertise in their knowledge of adult education (Q4) and research (Q5). 
Tests for all other items were two tail tests, since no hypothesis for direction of 
change was made. All tests used a significance level of .05. Zero differences were not 
included in the analysis. Coding for Questions 3, 4 and 7 (negative items) was 
reversed before analysis. 

Results 

The pre- and post-semantic differential surveys provide information about changes 
in the students' perceptions of various aspects of the delivery and the course. The 
other surveys (mid - completed by 80% of the students; reflect - completed by 67%) 
and student evaluations provided comments and further responses that enriched the 
picture. This section reports the results of the pre and post surveys and includes 
supplementary information from the other surveys where relevant. Tables 1, 2 and 3 
give z-scores arising from use of the Wilcoxon Signed Ranks test with the pre and 
post surveys. 



Table 1: z-scores for technology use (pre- and post-survey items) 





Q2 


Q6 


Q14 


z-score 


-1.84* 


-2.59* 


-0.31 
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Table 2: z-scores for course and content (pre- and post-survey items) 





Q3 


Q4 


Q5 


Q17 


Q18 


z-score 


-2.19* 


-1.82* 


-2.59* 


-0.34 


-1.15 



Table 3: z-scores for learning process (pre- and post-survey items) 





Q1 


Q7 


Q8 


Q9 


Q10 


Qll 


Q12 


Q13 


Q15 


Q16 


z-score 


-1.18 


-2.66* 


-1.07 


-0.05 


-1.01 


-1.78 


0 


-1:29 


0 


-2.25* 



For all tables * = p< .05. 



In the section that follows, items identified as showing significant change are 
presented in graphic form to indicate the direction of the change. Relevant 
information from other data sources is included. In all cases, four (4) is a neutral 
response. 



Technology use 

Some questions related to the use of technology are presented below: 
Q2: A range from a computer novice (1) to computer expert (7). 

Figure 1: Pre/post pairs Q2 




Q6: A range from apprehensive about internet use for the class (1) to confident about 
internet use for the class (7). 



Figure 2: Pre/post pairs Q6 




Course and content 

Related to internet use was a question that asked about exploring resources found on 
the web. Students in this course need to develop skills in locating and using 
resources for research purposes as well as foundational research skills. These are 
skills that they will use later in their program as they write their Masters papers. 

Q5: A range from research novice (1) to research expert (7). 

Figure 3: Pre/post pairs Q5 




Q1 in the reflect survey asked: How well do you feel you met the following course 
objective: 'Through this course, you will critically analyse research studies published 
in adult education journals for factors such as assumptions, biases, and contributions 
to the knowledge base and practice'. All students (100%) indicated they had either 
mostly or completely mastered the objective. 

Q5 in the reflect survey asked: How well do you feel you met the following course 
objective: 'Through this course, you will gain a foundation for your course in Adult 
Education and for writing your Masters paper in ADTED 588, which you will take 
toward the end of your degree program'. Ninety-four (94%) indicated they had 
mostly or completely mastered the objective. 

The other responses (Q 2, 3 and 4) from the reflection survey, which focused on 
course content mastery, indicate that most students rated their mastery of course 
objectives very highly. 

The mid survey provided some course- and content-related perspectives. At that 
point in the course, 80% of students were finding the course objectives easy to 
understand; 85% said the assignments and activities were clearly explained; and 95% 
said the assignments provided practice of the lesson objectives. The performance 
expectations and effort required were seen by 75% to be clear and feedback was 
timely (80%) and detailed (70%). 



The learning process 

The learning process, as already indicated, was a key consideration in the planning, 
delivery and teaching of this course. Two of the pre- and post-survey responses 
related to this concept. 

Q7: A range from prefer to be a self-directed learner (1) to prefer direction (7). 
Figure 4: Pre/post pairs Q7 




Q16: A range from dislike being helped in class by others (1) to like to be helped in 
class by others (7). 



Figure 5: Pre/post pairs Q16 
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In support of this finding, Q17 of the mid survey showed that 80% of the students 
interacted with other students between four and five times a week. Also from the 
mid survey, students indicated that they 'almost always or better' applied concepts 
in actual or simulated situations; felt 'very often or always' active participants in the 
teaching and learning process (90% of respondents); and 'very often or always' felt 
encouraged to challenge the instructor's and other students' ideas (75% of 
respondents). Most (60%) also felt they interacted with the instructor 'very often or 
always'. 

Data about time spent working on coursework and about working with others were 
gathered through the Time Tracking Chart and Q 18 in the mid-survey. At the mid- 
point of the course, 55% of the students indicated they spent 11-15 hours on the 
course and 20% indicated more than 15 hours. The Time Tracking Charts, when 
analysed at the end of the course, showed an average of 6.91 hours online and 6.26 
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hours offline per week over the whole course. The instructors also spent a total of 
approximately 12 hours per week on the course (both on and offline). 

There were two final questions in the mid survey. When asked if they would be 
willing to take another course that was designed in a similar manner, 100% of the 
students responded positively. When also asked if they would recommend another 
student to take a course designed in a similar maimer, 95% indicated they would. 

The students were given the opportunity to comment on various aspects of the 
course, such as group work, strengths and weaknesses, teamwork and the online 
community. Comments from these evaluations illustrate some of the discussion 
points that follow. 

Discussion 

This section comments briefly, in turn, on each of the three areas that formed the 
basis of the pre and post surveys. In particular, however, it draws on students' 
comments about one aspect of the learning environment - working in groups. 



Technology use 

In two of the items related to this theme, significant change can be seen. As might be 
expected in a course that requires considerable interaction via computer, students 
reported more expertise in computer use and less apprehension about the use of the 
internet for the class. 

Course and content 

The responses to Q5 (pre/ post survey) about research novice/ research expert show 
that while most wouldn't call themselves experts, they did become more confident. 
The learning objective related to using research studies (Q1 reflect survey) and the 
one related to using their research skills (Q5 reflect survey) showed very high levels 
(100% and 94% respectively) of confidence appropriate to their level of study. 



The greatest strength of the course for the students was the instructors and the 
guidance and support provided. Interaction was the second greatest strength. 

Strengths: Flexibility of timing, well thought out and planned, ease in use 
once you get started, availability of support, positive support from 
instructors, printed material and course is user friendly, non-intimidating 
(most of the time). (Comment from student 24). 



The learning process 

The results showed that the learning environment created made good matches to the 
learning principles that had guided the delivery design. Computer conferencing can 
allow the interactive and collaborative aspects of learning to flourish. As has been 
indicated, many student comments pointed to the importance of the instructor in the 
learning process. 

Strengths: positive support from instructors. (Comment from student 8). 
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Of the eleven items that contributed to the learning process theme, two were found 
to be significant. First though, note the change reported in the previous course and 
content section that students grew to feel more capable of using the internet to search 
for and find information useful in the course. While this item also reflects the 
confidence in technology use noted already, it must reflect more than that. Students 
reporting a shift to expertise must also be weighing their ability to make judgments 
about the nature of the material useful in their particular learning environment. This 
increasing expertise is matched by a significant shift in feelings towards self direction 
in learning (Q7). These changes point to the increasing independence of students. 
This conclusion is supported by the results of Q16 that students grew to dislike being 
helped. These points do not, however, indicate that students did not enjoy or 
appreciate working together. 

The evaluative comments gathered about group work enriched the picture of student 
interaction and support that emerged. These supported the literature findings 
suggesting that, for distance students, the learning process can be enriched through 
small group work. Group work was important for the students and a valuable aspect 
of the course. Ninety-three percent (93%) of the students who responded mentioned 
that they had found the group work valuable. Those who elaborated on this 
mentioned the sense of community or classroom that was built and the value and 
support they had found in this. It was also evident that most groups and 
partnerships had worked well. 

It [the group work] gave me the feeling that I was actually in a classroom 
environment (Comment from student 17.) 

I (we) thought the group work was the best part of the class. It really gave 
me that much needed human contact and sounding boards. (Comment 
from student 23) 



I think it [group work] did establish a learning community and for me, it 
added some discipline and structure to an assignment. Peer pressure is 
useful for procrastinators. (Comment from student 13). 



I found that we worked so well together that I would have preferred all 
assignments to be done that way. (Comment from student 9). 

The students also commented on group size, indicating that a group of six was the 
maximum size they would want. They had found the amount and timing of group 
work to be about right but experienced some difficulties getting together - even 
online. 

There was further support for the group work and the development of an online 
community in the comments associated with the reflect survey. Over ninety percent 
of the students indicated they felt part of an online community. Most also 
commented on the value of shared contributions that had helped them develop the 
course concepts. It had been, they indicated, a valuable and enjoyable experience. 
Clearly, the students grew to increasingly value independence and interdependence, 
but not dependence. 



I really felt like part of a class with this experience. With the exception of 
knowing what people looked like I feel I got to know personalities, 
interests and levels of experience in Adult Education. We interacted a lot 
which I found an interesting experience. I enjoyed the web pages that we 
had to do during that first week. Great idea. I really feel like part of a 
community. (Comment from student 6). 



Conclusion 

While the study demonstrated that an appropriate online learning environment 
could be developed for distance adult students, we should not consider that the 
move from traditional class teaching to online is without questions and mysteries of 
its own. There are areas to research that would further develop our understanding of 
online teaching and learning and benefit the new and returning students who are 
moving into higher education. We need to know more about how students interact 
with the material, how groups work to support learning, and the role the instructor 
plays in facilitating and developing learning in a collaborative distance environment. 
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